Selectively distributing updates of changing images to client devices

ABSTRACT

A network server for sharing images with mobile electronic devices having one or more processors, and image sharing client application, and an image distribution application. The image sharing client application is to be executed on one or more of the processors to receive an update to an image being shared over a network with mobile electronic devices that each have their own current zoom level and their own currently visible area of the image. The image distribution application is to be executed on one or more of the processors to keep a record of the current zoom level and currently visible area for each of the plurality of mobile electronic devices and to selectively send image updates to the plurality of mobile electronic devices based on the current zoom level and currently viewable area of the mobile electronic devices.

FIELD OF THE INVENTION

The present invention relates to sharing changing images over a networkwith client devices, and in particular to the distribution of dynamicimages from a remote network server over a network to client devices.

BACKGROUND OF THE INVENTION

There has been a great increase in the use of mobile electronic devices(e.g., mobile/cellular/smart phones, Personal Digital Assistants (PDAs),microcomputers, handheld computing and gaming devices and the like).These mobile electronic devices have become increasingly complex. Thenumber and type of applications that these mobile electronic devicessupport and implement is large and will likely continue to increase. Oneclass of applications involves continuous distribution over a cellularnetwork of full or partial images from an image host to a plurality ofmobile client devices that display these images.

In order to promote their mobility, the size of these mobile electronicsdevices is relatively small compared to conventional laptop and desktopcomputers, a constraint that also limits the size of their displayscreen. To operate applications on these mobile electronics devices, theuser may rely on image navigation functions, such as zooming andpanning, to locate and show details of a certain partial area of a largeimage. To allow for zoom-in activities by the users, mobile systems maydownload images in high-resolution.

Mobile device users may view and interact with dynamically changingimages provided by a remote server. Examples of this class ofapplications include and remote desktop applications, in which updatesof changing screen images, in full or partial image form, are beingdynamically sent from the server to be received and displayed on theclient devices. For example, the image may be a slide showing a piechart that is being played on an image host that is connected to theremote network server. This image may change when the user of the imagehost moves her cursor to another part of the screen, in which case onlypart of the image changes, or moves to a new slide, in which case theentire image may change. The image distributed to the mobile device maybe said to be dynamic if the image may change over time.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a network server forsharing images with mobile electronic devices, the network processorhaving one or more processors, and image sharing client application, andan image distribution application. The image sharing client applicationmay be executed on one or more of the processors to receive an update toan image being shared over a network with mobile electronic devices thateach have their own current zoom level and their own currently visiblearea of the image. The image distribution application may be executed onone or more of the processors to keep a record of the current zoom leveland currently visible area for each of the plurality of mobileelectronic devices and to selectively send image updates to theplurality of mobile electronic devices based on the current zoom leveland currently viewable area of the mobile electronic devices.

For a better understanding of the present invention, together with otherand further features and advantages thereof, reference is made to thefollowing description, taken in conjunction with the accompanyingdrawings, and the scope of the invention will be pointed out in theappended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates a system for distributingupdates of changing images to mobile devices according to embodiments ofthe present invention.

FIG. 2 is a block diagram of an image host displaying an image, and amobile communications device displaying part of that image, according toembodiments of the present invention.

FIG. 3 is a block diagram of a network server according to embodimentsof the present invention.

FIG. 4 is a block diagram that illustrates an exemplary textual formatof the current viewing status data, and the corresponding relativeposition of a current viewing area and full image, according toembodiments of the present invention.

FIG. 5 is a block diagram that illustrates distribution of images to aclient using a combination push-and-pull method according to embodimentsof the present invention.

FIG. 6 is a flow chart that illustrates the push method of distributingimages to clients according to embodiments of the present invention.

FIG. 7 is a flow chart that illustrates the pull method of distributingimages to clients according to embodiments of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

It will be readily understood that the components of the presentinvention, as generally described and illustrated in the figures herein,may be arranged and designed in a wide variety of differentconfigurations in addition to the described presently preferredembodiments. Thus, the following more detailed description of theembodiments of the present invention, as represented in the figures, isnot intended to limit the scope of the invention, as claimed, but ismerely representative of selected presently preferred embodiments of theinvention. The following description is intended only by way of example,and simply illustrates certain selected presently preferred embodimentsof the invention as claimed herein.

Embodiments of the present invention are directed to the distributing ofupdates of changing images to mobile devices with relatively smallscreens, such as personal digital assistants or cellular phones. Theinvention provides a method that allows users of applications on mobiledevices to view shared image updates with less network delay for animproved user experience. Embodiments of the invention broadlycontemplate improved ways of distributing images to mobile electronicdevices, with the images being dynamically distributed from a networkserver. For convenience, the applications requiring the distribution ofchanging images to mobile electronic devices are referred to asapplications with dynamically changing images. The improvements of thisinvention is best manifested on changing image applications thatdistribute high-resolution images (e.g., 1400×1040 pixel resolution) tomobile electronic devices with a relatively small screen size in termsof displayable pixel resolution (e.g., 480×320 pixel resolution).

To view visual details in high resolution from a small mobile devicescreen, the user may use zooming and panning functions to navigate andlocate detail information on the large images. By zooming in, thesmaller screen of a mobile client covers only a smaller area of theentire image. By panning around, the area of the image that is currentlyshown on the display of the mobile client is changed. Therefore, in asystem in which a plurality of mobile clients is sharing a highresolution image, each of these mobile clients may display a differentarea of the image. Since the small screen of a mobile device may at anytime cover only a partial portion of the full image shared by the imagehost, the new updates of the shared image may fall either inside oroutside of the area covered by a mobile device's display screen. Inembodiments of the invention, only the updates that fall inside a mobiledevice's display screen are sent to that mobile device. As a result, thesize of data transmission to update the mobile device screen can besmaller than that of the approach in which all updates are transmittedto the mobile device. Thus, the user may experience a faster responsetime in viewing changes to images shared by the remote image host.

Embodiments of the invention provide an application at a network serverthat receives new images (which can be full images or partial updateimages) from an image host (such as a remote desktop host computer or acomputer whose screen image is to be shared, or their respective proxyservers). According to this embodiment, the network server thendetermines, for each mobile client, the respective areas within thereceived new images that are to be sent to each of those mobile clients.In embodiments of the invention, the factor that determines what areawithin update images is to be sent to a specific mobile client is basedon the visible area on the full image that the display screen of themobile client currently covers.

The zoom level aspect of the invention provides a method for the clientto retrieve update images at the resolution that is adequate for thesmall mobile device screen while improving the update image size, hencereducing the network delay. A higher zoom level is associated with imagewith a higher resolution. In embodiments of the invention, the clientdevice receives the initial shared image with the viewing area coveringthe entire image but at the lowest zoom level (i.e., zoom level=0). Theclient device increases the zoom level when the user of the clientdevice zooms in to view more details of the image. By zooming in, theclient device requests the image with a higher resolution but with asmaller view area because the latter contracts within the full imagerelative to the size of the full image. At any given time, the imagestransmitted are either lower resolution images covering a larger viewarea or higher resolution images covering a smaller view area, reducingthe need to transmit image data with an excessive size (i.e., highresolution images covering a large view area). For example, the size ofthe data that represents a typical PowerPoint® slide image may rangefrom approximately 133 Kilobytes (KBs), at a resolution of 1150×860pixels, to approximately 66 KBs, at a resolution of 720×540 KBs.

In some embodiments, a network gateway server optimizes network load forany given image retrieval, hence allowing the user to view and interactwith high-resolution dynamically changing images provided by a remotehost in a more fluid manner. A server agent at the network server maykeep track of each client device's current zooming level and visiblearea within an image. An image distribution agent at the server may thenupdate an images on a client device by sending the recent changes of theimage that are based on the currently viewable area and the zoom levelof the image on the client. In some embodiments, the originalhigh-resolution images are transformed into different levels ofresolution because images with less resolution are smaller in size andhence can be transmitted to the devices faster. When the system receiveschanged image tiles from the image hosts, it may transform them intoaforementioned different levels of resolution. Using methods of thisdisclosure, the mobile client only downloads images with resolutionexactly or closely matched to that of it screen, hence eliminating theneed to always download high-resolution images which incurs long networkdelay.

In embodiments, for each client, the system keeps track of its currentviewing resolution level and the viewing area (also know as the viewingbox) mapped onto the target image of the same resolution level. When thesystem detects new image changes, it may send to each client only thechanges of the same resolution level and that are within the client'sviewing box. When the user performs the zooming or panning function, thenew parameters containing the latest resolution level and viewing boxare sent to the remote network server.

According to these embodiments, the gateway system takes client requeststo connect to an image host that provides real-time dynamically changingimages (e.g., screen sharing). Upon receiving each dynamic image fromthe image host, the gateway performs transformation to createcorresponding images each with a different level of resolution andmaintains a connection to each client allowing for both push and pullfunction. The push function allows the gateway to push new changes tothe clients whereas the pull function allows the client to store itsstate on the gateway. The status of a client device may be its currentlyviewing resolution level and the viewing area within the image of thecurrent resolution level. When the gateway receives new image changesfrom the image hosts, it may perform the push function for each clientwaiting for the changes. For each waiting client, the gateway mayretrieve the latest viewing resolution level and viewing area and onlysend the changes of the same resolution level that are within theviewing area to the client. If a change image spans across the viewingand non-viewing area for a client, the gateway may clip it so that thepart that is within the viewing area is sent.

FIG. 1 is a block diagram that illustrates a system for distributingupdates of changing images to mobile devices according to embodiments ofthe present invention. As shown, a network server 120 is coupled to anexternal image host 110 and to a network 130, which may for example be acellular communications network, an intranet, the public Internet, orsome combination of these. As shown in FIG. 1, network 130 includes aplurality of communication devices 137-139 (which may be mobileelectronic devices such as a mobile/cellular/smart phones, PDAs,microcomputers, handheld computing and gaming devices and the like). Forexample, devices 137-139 may be a BlackBerry® wireless handheld devicemade by Research In Motion and/or a Palm Treo® smartphone made by Palm,Inc. Network server 120 may be referred to as a gateway server orgateway network server as it separates the image host from the outsidenetwork.

Network server 120 may be any type of computing device which managesnetwork resources, such as a personal computer, a workstation, a laptopcomputer, etc. Image host 1 10 may be any type of computing device thathosts an application that projects images, such as a screen sharing orremote desktop application, in which updates of changing screen images,in full or partial image form, are being dynamically sent from theserver to be received and displayed on the clients. Network 130 may beany type of network and may include an intranet, the public Internet, acellular network, a local area network, a wide area network, a virtualprivate network, etc. While in some embodiments server 120 connects toexternal image host 110 through a high-speed network connection, inother embodiments both server 120 of the current invention and an imagehost 110 may reside in the same computing device. Alternatively, it isalso applicable that the image host 110 may communicate with the server120 of the current invention through a third proxy server.

In the example shown in FIG. 1, image host 110 is displaying an image 10that contains a chart 9, such as a pie chart that is on a slide of aLotus® Freelance® or Microsoft® PowerPoint® presentation. For thepurposes of illustration, image 10 is shown as divided into four imagetiles. More typically, image 10 would be divided into a larger number oftiles, such as 1452 tiles arranged in a grid of 44 by 33. In embodimentsof the invention, image host 110 sends image updates to network server120 when the image 10 being displayed at image host 110 is changed by auser of image host 110, such as a person who is controlling thepresentation. The image updates can be in the form of image tiles eachof which is a small rectangle or square area of the original full image.The very first full image sent by the image host 110 to the server 120may be modeled as image updates containing all image tiles of the fullimage. As would be appreciated by a person of skill in the art, theimage may be stored and transmitted as a matrix of pixels or using thePortable Network Graphic (PGN) format or the MPEG-3 or MPEG-4 formats.

As depicted in FIG. 1, the server 120, upon receiving an update imagesfrom image host 110, creates and stores multiple image versions 10 a ,10 b , 10 c , each with a different zoom level (0, 1, 2 respectively).These images may be stored in a memory of network server 120 andrepresent various versions of the image 10 each at a different zoomlevel. In embodiments of the invention, the image versions 10 a-10 cthat are stored in network server 120 are updated when image host 110sends server 120 an indication that image 10 was updated. An example ofan image update may be a change to a presentation being displayed at theimage host device in screen sharing mode. In the embodiment shown, theimage 10 a with zoom level 0 represent the original image, as updated,received by server 120 from the image host 1 10. The image update 10 bwith the next zoom level 1 represent update images with resolution thatis one notch lower than 10 a , whereas the image updates 10 c with zoomlevel 2 represent update images with resolution that is one notch lowerthan 10 b . As an example, zoom level 0 may be at a resolution of480×320 pixels, level 1 may be at 600×400 pixels, and level 2 may be at720×480 pixels. The number of zoom levels and the difference ofresolution between two adjacent zoom levels may be preconfigured for aspecific embodiment of the current invention. In the preferredembodiment of the current invention, the various versions of imageupdates 10 a -10 c can be stored in main memory for maximum storing andretrieving efficiency. Alternatively, they can be stored in disks, butthe storing and retrieving efficiency will suffer.

As exemplified in FIG. 1, client mobile devices 137-139 may be connectedto the server 120 through network 130, which as discussed above may be acellular network connection. While FIG. 1 shows three example clientmobile devices 137-139, the actual number depends on how many clientmobile devices have installed the client application and how many ofthem are connected to the server 120 at any given time. As shown in FIG.1, each of the mobile devices have a screen 147-149 that may be used todisplay an image such as image 10.

In the embodiment shown, network server 120 includes an imagedistribution application 140 which may be a software application thatexecutes on a processor of network server 120. Image distributionapplication 140 may receive an image from image host 110 that is to beshared with devices 137-139 and may send that image to those devices asnetwork messages 107-109. Image distribution application 140 may thenreceive an update to that image from image host 110 and may send thatimage update to those devices, also shown in FIG. 1 as network messages107-109. In addition, the mobile devices 137-139 may send their currentviewing status data to the server 120 as network messages 127-129. Thecurrent viewing status data may be contained in an Extensible MarkupLanguage (XML) file. Upon receiving a change to the respective currentviewing status data from mobile devices 137-139, the server 120 stores(i.e., records) that for future retrieval as current viewing areaparameters 125. When the server 120 receives new image updates fromimage host 110, server 120 creates multiple versions 10 a -10 c of thenew image updates to reflect the different resolutions may be used bydevises sharing the image. Server 120 then, based on the stored currentviewing area parameters 125, determines respectively the image updatesof the current zoom level of mobile devices 137-139 that are within thecurrent viewing area of these mobile devices. After the appropriateimage updates for each mobile client is determined, the server 201 sendsthem to each mobile client for display.

FIG. 2 is a block diagram of an image host displaying an image, and amobile communications device displaying part of that image, according toembodiments of the present invention. FIG. 2 shows a simplified view ofa mobile electronic device 137, such as a mobile/cellular/smart phoneand the like, and includes further details of mobile electronic device137 of FIG. 1. The mobile electronic device 137 comprises structuralcomponents, such as those described in FIG. 1, necessary to carry outthe functionality as herein described. As is apparent from thedescription, the present invention is applicable to any appropriatelyconfigured electronic device, as described herein.

In the embodiment of FIG. 2, and as discussed with reference to FIG. 1,mobile electronic device 137 may communicate with network server 120,which may in turn communicate with image host 1 10. As also discussedabove, image host 110 is shown in FIG. 2 as displaying an image 10 thatincludes a chart 9. Parts of chart 9 appear in different quadrants, ortiles, of image 10, so that approximately a quarter of chart 9 appearsin each of the four quadrants of image 10.

Mobile electronic device 137 of FIG. 2 is shown as including a display147, a processor 220, a mobile communications module 230, an imagenavigation application 240, and an image distribution client 250.Processor 220 may be a circuit that processes instructions that drivethe operation of device 137, and may be for example a processor thatconforms to the ARM 32-bit RISC instruction set architecture (ISA)developed by ARM Limited. Mobile communications module may includesoftware and/or hardware that is used to communicate over a wirelessnetwork with a cellular station. Though not shown, device 137 may alsoinclude a user interface which may be any of a wide variety of userinterfaces, including but not limited to a keypad or keyboard, touchpad, etc. The user interface may also be integrated with the displayscreen 147, such as with a popular “touch screen” implementation.Communications module 230 facilitates transmission and reception ofnetwork communications, such as frequent image updates, as describedherein. Accordingly, the mobile electronic device 137 is capable ofusing an operating system and appropriate hardware components configuredfor communicating with remote devices via a network connection (e.g.wireless Internet connection) and downloading, displaying, manipulatingand modifying pages, as described herein.

Image navigation application 210 may be software that allows a user ofdevice 137 to navigate within an image that is displayed on display 147,such as by panning or zooming. Image navigation application 210 may bepart of a general user interface application that allows a user tointerface with applications running on device 137. Image distributionclient 250 may be software that provides current viewing area parametersfor device 137 to network server 120, and in particular to imagedistribution application 140. For example, when the user of device 137pans or changes the zoom for the currently displayed image, imagedistribution client 250 may provide network server 120 with the newcurrent viewing area parameters. In response, network server 120 maystore the current viewing area parameters for device 137 in its databaseand may send the part of the image being panned to, or an image with thedesired new resolution, to device 137. Image navigation application 240and image distribution client 250 may be software that is executed byprocessor 220.

As shown in FIG. 2, display 147 of mobile electronic device 137 isdisplaying partial chart 9 a . In this example, partial chart 9 a is theupper right part of chart 9 that is shown in image 10 at image host 110. In this example, the user of device 137 may have been viewing image10 on display 147 and then may have zoomed in to see details of theimage and panned over to the upper corner so that only partial chart 9 ais shown on display 147. In addition to the features shown in FIG. 2,mobile electronic device 137 may also include additional elements suchas additional processors, memories such as a Read-Only Memory (ROM) anda Random Access Memory (RAM), and one or more busses.

FIG. 3 is a block diagram of a network server according to embodimentsof the present invention. In particular, FIG. 3 shows further details ofnetwork server 120 of FIG. 1. As shown in FIG. 3, network server 120contains processor(s) 320, a memory 330, a presentation application 315,an image sharing application 317, an image distribution application 140,a graphics adapter 360, a display 365, and a network communicationmodule 350. System processor(s) 320 may be one or more general-purposeprocessor(s) and may comprise one of the processors produced by IntelCorporation or by International Business Machines Corporation.Processor(s) 320 may be coupled to a ROM and a system memory (RAM) by aprocessor bus and may execute application programs under the control ofoperating system and application software stored in system memory.Network server 120 may also include a number of other devices, includingbusses, adapters and bridges, such as a network adapter, which mayinterfaces to a Local Area Network that is in turn coupled to othernetworks such as a cellular network and/or the Internet. In FIG. 3,network communication module 350 includes software and/or hardware forcommunications between network server 120 and external networks, such asnetwork 130 of FIG. 1.

Network server 120 may also include a graphics adapter 360 thatinterfaces network server 120 to display 365. As shown in FIG. 3,display 365 is displaying the image 10 of FIG. 1. Network server 120 mayinclude one or more input ports for communicating with networks such asnetwork 130 and with devices such as image host 110. Network server 120may also include an input/output (I/O) controller, which controlscommunication between network server 120 and attached peripheral devicessuch as a keypad, touch pad, and a disk drive. In addition, an I/Ocontroller may be included that supports external communication bynetwork server 120 via serial and parallel ports.

As shown in FIG. 3, memory 330 stores Current Viewing Area Parameters125 and Image Versions 10 a-10 x , which are described above withreference to FIG. 1. Memory 330 may be any type of computer readablestorage medium such as a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), a portablecompact disc read-only memory (CD-ROM), an optical storage device, amagnetic storage device, or any suitable combination of the foregoing.In embodiments, presentation application 315, image sharing application317, and image distribution application 140 are comprised of softwareinstructions are stored on memory 330 and may be executed byprocessor(s) 320. Presentation application 315 may be an application,such as Lotus® Freelance® or Microsoft® PowerPoint®, that presentsimages that may be viewed on a display screen. Image sharing application317 may be an application, such as Lotus Sametime Unyte™ or Microsoft®NetMeeting®, that allows one user to share that user's screen with otherparticipants in a session so that all the participants in a meeting canview the same presentation on their respective display screenssimultaneously. While the embodiment of FIG. 1 has an image host 110that is external to the network 120, for the purposes of illustrationthe embodiment shown in FIG. 3 has the image host's functionality (i.e.,the functionality of Presentation application 315 and image sharingapplication 317) are included in the network server. As discussed above,image distribution application 140 may be a software application thatreceives an image to be shared with remote devices and sends that imageto those devices as network messages. Image distribution application 140may for example carry out the server functionality described withreference to FIGS. 5-7 below.

In a preferred embodiment of the invention, the update images receivedfrom the image host represents the highest resolution. Upon receivingimage updates, various versions of the updates are created, each with adifferent resolution represented by a different zoom level. When theuser zooms in on an image, the client application increases the zoomlevel. A higher zoom level is associated with images with higherresolution. For example, when the client first receives the sharedimage, the viewing area is the entire image while the zoom level is 0(meaning no zooming in yet). Because the zoom level is 0, the clientreceives the entire image with the lowest resolution. The lower theresolution of an image, the smaller is the size of the image.

FIG. 4 is a block diagram that illustrates an exemplary textual formatof the current viewing status data, and the corresponding relativeposition of a current viewing area and full image, according toembodiments of the present invention. FIG. 4 shows exemplary currentviewing status data 401 in the XML format. While the XML format is shownhere, a person of skill in the art would appreciate that other formatsmay be used, such as JavaScript Object Notation (JSON). As depicted inFIG. 4, the information contained in the current viewing status data 401includes the currently viewable area modeled by four integers, namelythe xOffset=60, yOffset=40, width=480, and height=320. The xOffset andyOffset in the example in FIG. 4 represent x-axis and y-axis pixeloffset from the upper left corner of the current viewing area on thefull map. The width and height are the width and height of the viewingarea in number of pixels. Other information included in the currentviewing status data example 401 includes the zoomLevel=2 and theseqNumber=25. The zoomlevel integer represents the current zoom levelwhereas the seqNumber represents the sequence number used by the networkserver 120 to determined when the current viewing area at the currentzoom level were last updated.

The full image 402 in FIG. 4 shows the resolution (720×480) of the fullimage at zoom level=2. The rectangle 403 in FIG. 4 depicts the currentlyviewable area that is modeled inside the XML representation of thecurrent viewing status data 401. If the user performs a panningfunction, the currently viewable area 403 will move to a differentlocation within 402. If the user performs a zoom in function, thecurrently viewable area will display images with zoomlevel=3, which hasa higher resolution. The currently viewable area 403 contracts in itssize relative to the new images with zoomLevel=3. If the user performs azoom out function, the current view area will display images withzoomLevel=1, which has a lower resolution. The currently viewable area403 expands in its relative to the new images with zoomlevel=1.

FIG. 5 is a block diagram that illustrates distribution of images to aclient using a combination push and pull method according to embodimentsof the present invention. As shown in FIG. 5, the push 530 methodembodies one feature of the current invention in which the networkserver 120 dynamically sends image updates 107 (as shown in FIG. 1) tothe client device 137 responsive to host image changes and based on thecurrent viewing status data of client device 137. In order for the pushmethod to be functional, the network server 120 typically keeps a liveconnection with the client device 137. This is because in existingclient-server systems, such as in the World Wide Web (WWW) using theHTTP or HTTPS protocols, the server does not keep the network addressinformation of the client and the client does not accept connectionrequests from other hosts, such as the server. Should the server losethe live connection with the client, the server cannot activelyestablish another connection with the client. It is up to the client toreconnect to the server to establish a new live connection.

However, in existing open client-server systems such as WWW, aclient-server connection cannot be kept alive by the serverindefinitely, due to resource constraints, and will expire after acertain amount of time. To remedy this limitation, a preferredembodiment of the current invention that uses WWW open networktransportation protocol (i.e., HTTP) can deploy the “long poll”technique to keep the push connection alive. The long poll technique isdescribed, for example, inhttp://www.perplexedlabs.com/2009.05/04/php-jquery-ajax-javascript-lomg-polling/,a copy of which is herein incorporated by reference. Using the long polltechnique, the client opens a Hyper Text Transfer Protocol (HTTP)connection with the server. The server keeps the HTTP connection for acertain amount of time. Right before the HTTP connection expires, ifthere is no new image updates available for the client, the server sendsa response back to the client asking the client to open another HTTPconnection again. Thus, other than during the maintenance time when theserver responds to ask for a new connection, the server always has alive connection with the client, during which time any new image updatescan be pushed to the client. When the client receives new image updatesor the open connection response from the server, the client opens a newconnection with the server again to establish a new push link.

FIG. 5 also depicts a pull 540 method in which the client device 137sends its new current viewing status data to the network server 120.Once the new current viewing status data message 127 of FIG. 1 isreceived by the network server 120, the server may determine that newimage updates are available based on the new viewing area and zoomlevel. In that case, new image updates 107 are then sent from the serverto the client. For the pull method, the server needs not maintain a liveconnection with the client for any extended period of time. Regular HTTPGet or Post method calls can be used to accomplish this task.

FIG. 6 is a flow chart that illustrates the push method of distributingimages to clients according to embodiments of the present invention. Asshown in FIG. 6, the server receives new image updates from the imagehost in step 610. Next, in step 620, the server creates multipleversions of the received image updates with different resolutions eachdenoted by a unique zoom level. In the next step 630, the serverretrieves the current viewing status data for all mobile clients thatare connected to the server. Next, in step 640, for each mobile client,the server determines the eligible image updates at the same zoom levelas the client's current zoom level, such that only image updates withinthe client's current viewing area are to be used. For example, the imageupdate might not be sent to a client device if it is determined that thereceived image update does not indicate a change to the image beingdisplayed in the currently visible area for that client device. Theimage updates that the sever determines to be appropriate are then sentto the respective clients in step 650. The new image update may beobtained from a version of the image that is stored at the networkserver and selected based the updated parameter. The update may be sentas fragments of the image (i.e., image tiles), with only those fragmentsbeing sent that represent changed parts of the image that are within thecurrent viewing area of the client device. In embodiments, if it isdetermined that the received image update represents a change to two ormore fragments with overlapping areas or adjacent boundaries in theimage being displayed in the currently visible viewable area for one ofthe client devices, then the image update may be sent to the clientdevice as a merged fragment that represents the two or more fragmentswith overlapping areas or adjacent boundaries. On the client side, uponreceiving the new image updates from the server in step 660, the clientthen displays them accordingly in step 670.

FIG. 7 is a flow chart that illustrates the pull method of distributingimages to clients according to embodiments of the present invention. Asshown in FIG. 7, the client first detects if the user has invoked azooming or panning action (step 7 1 0). Upon detecting the zooming orpanning action, the client then calculates the new current viewingstatus data (step 720). A panning action changes the current viewingarea but not the zoom level. A zooming action may change both the zoomlevel and the current viewing area because the xOffset and yOffset(depicted in FIG. 4) may change on the new image with the different zoomlevel.

After the new current viewing status data is calculated in step 720, theclient sends it to the server in step 730. Upon receiving the newcurrent viewing status data sent from a client (step 740), the serverupdates it in the server's internal repository (step 750). Next, theserver determines the eligible image updates at the same zoom level ofthe newly received current viewing status data, such that the determinedimage updates includes only the image updates within the client'scurrent viewing area (760). The determined image updates are then sentto the client in step 770.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code (i.e., software)embodied thereon, such as memory 330 shown in FIG. 3.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electromagnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing. Computer program code for carrying out operations foraspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language such as Java, Smalltalk, C++ or the like andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The invention provides a method of adaptive distribution and viewing ofimage updates for dynamically fast changing images sent by a remoteimage host. Embodiments of the invention detect the latest viewing areastatus, based on which image updates are filtered such that only imageupdates within a client's current viewing area are sent to the client.Such filtering helps reduce the size of image objects that are requiredto transmit to the client in real-time. The invention also createsmultiple versions of image updates with various resolutions such thatwhen viewing image covering a large area on a small screen, imageupdates with low resolution are sent to reduce real-time transmissionload without the loss of legibility. Another aspect of the inventionprovides a method comprising: receiving the information of the currentzoom level of each client; receiving from an image host the latestupdates of the image shared by the clients; creating various versions ofthe update images at different respective resolutions, eachcorresponding to a predefined zoom level; and sending to each clientonly the portion of the version of updates that is within the currentviewing area and of the same current zoom level of the each client.

Reference throughout this specification to “one embodiment” or “anembodiment” (or the like) means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, appearancesof the phrases “in one embodiment” or “in an embodiment” or the like invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the described features,structures, or characteristics may be combined in any suitable manner inone or more embodiments. In the following description, numerous specificdetails are provided to give a thorough understanding of embodiments ofthe invention. One skilled in the relevant art will recognize, however,that the invention can be practiced without one or more of the specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures, materials, or operations are not shownor described in detail to avoid obscuring aspects of the invention. Itis to be understood that the invention is not limited to those preciseembodiments, and that various other changes and modifications may beaffected therein by one skilled in the art without departing from thescope or spirit of the invention.

1. A network server for sharing images with mobile electronic devicescomprising: one or more processors; an image sharing client applicationto be executed on one or more of the processors to receive an update toan image being shared over a network with a plurality of mobileelectronic devices that each have their own current zoom level and theirown currently visible area of the image; and an image distributionapplication to be executed on one or more of the processors to keep arecord of the current zoom level and currently visible area for each ofthe plurality of mobile electronic devices and to selectively send imageupdates to the plurality of mobile electronic devices based on thecurrent zoom level and currently viewable area of the mobile electronicdevices.
 2. The network server of claim 1, wherein the server furthercomprises a memory to store a plurality of versions of the image,wherein each of these versions has a different resolution level.
 3. Thenetwork server of claim 2, wherein the memory stores data indicative ofthe currently visible area and zoom level for each of the plurality ofmobile electronic devices.
 4. The network server of claim 1, wherein theserver further comprises an input port to receive updates to the imagefrom an image host device.
 5. The network server of claim 4, wherein theimage update is a change to a presentation being displayed at the imagehost device in screen sharing mode.
 6. The network server of claim 1,wherein the network is a cellular telephone network and each of theplurality of client devices is a mobile cellular communications device.7. A method for sharing images with client devices, the methodcomprising: receiving, at a network server, an update of an image beingshared with a plurality of client devices that are coupled to the serverover a network, wherein each of the client devices has a display thatshows a currently viewable area that contains all or a selected part ofthe image being shared; determining at the network server, based onrecorded parameters of the currently visible area for each of theplurality of client devices, whether the received image update indicatesa change to the image being displayed in the currently visible area foreach of the plurality of client devices; and sending the image updateselectively to one or more of the client devices based on the results ofthe determination.
 8. The method of claim 7, wherein the image update isnot sent to a client device if it is determined that the received imageupdate does not indicate a change to the image being displayed in thecurrently visible area for that client device.
 9. The method of claim 7,wherein the image being shared with the plurality of client devices iscomprised of a plurality of fragments, and wherein if it is determinedthat the received image update represents a change to two or morefragments with overlapping areas or adjacent boundaries in the imagebeing displayed in the currently visible area for one of the clientdevices, then the image update is sent to the client device as a mergedfragment that represents the two or more fragments with overlappingareas or adjacent boundaries.
 10. The method of claim 7, furthercomprising the steps of: receiving at the network server an update froma client device to a parameter of the currently visible area for thatclient device; storing the received parameter update at the networkserver; and sending a new image update to that client device based onthe updated parameter.
 11. The method of claim 8, wherein the new imageupdate is obtained from a version of the image that is stored at thenetwork server and selected based the updated parameter.
 12. The methodof claim 7, wherein the network server stores parameters of thecurrently visible area for each of the plurality of client devices thatinclude the resolution level for the client device and an identificationof the part of the image that is visible at the client device.
 13. Themethod of claim 7, wherein the step of sending image updates from theserver to the client devices is based on the long polling technique. 14.A method for processing a shared image on a mobile electronic devicecomprising: displaying on a screen of a mobile electronic device animage received from a network server over a network, wherein the imagedisplayed at the mobile electronic device represents the currentlyvisible area of an image maintained at the network server; receiving achange from a user of the mobile electronic device to one or moreparameters of the visible area for the mobile electronic device; andsending an update to the network server of the current parameters of thecurrently visible area for the mobile electronic device.
 15. The methodof claim 14, further comprising receiving an image update from thenetwork server over the network, wherein the image update corresponds tothe change to the currently visible area of the mobile electronicdevice.
 16. The method of claim 14, wherein the change to the currentlyvisible area that is received from the user is one of a zoom into thedisplayed image or a pan to a part of the image maintained at thenetwork server that is different than the image currently displayed atthe mobile electronic device.
 17. A computer program product for sharingimages with mobile electronic devices, the computer program productcomprising: a computer readable storage medium having computer readableprogram code embodied therewith, the computer readable program codecomprising: computer readable program code configured to keep a recordof the current resolution level and currently visible area of aplurality of client mobile electronic devices that are sharing an image;computer readable program code configured to detect a change to theshared image and, in response, to send to each client mobile electronicdevice only the changes of the same resolution level and that are withinthe client electronic device's currently visible area; and computerreadable program code configured to receive parameters containing a newcurrent resolution level or currently visible area from a client mobileelectronic device.
 18. The computer program product of claim 17, whereinthe computer readable program code further comprises computer readableprogram code to determine, based on the received parameters containing anew current resolution level or currently visible area for a clientmobile electronic device, that an image update should be sent to theclient device and to send that image update to the client device. 19.The computer program product of claim 17, wherein the computer readableprogram code further comprises computer readable program code to onlysend changes to the client mobile electronic devices that are of thesame current resolution level as the client mobile electronic device andthat are within the currently visible area of the client mobileelectronic device.
 20. The computer program product of claim 17, whereinthe computer readable program code further comprises computer readableprogram code to determine if a changed image spans across the currentlyvisible area and non-viewing area for a client and, if so, to clip theimage so that only part of the image within the currently visible areais sent to the client mobile electronic device.